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ABSTRACT 

The purpose of this study was to use secondary 
analysis of an Australian database known as the Second International 
Science Study (SISS) to examine the role of student, school and home 
factors in explaining student differences in science achievement, 
especially on sex differences. Student characteristics investigated 
in the study included home background, attitude towards science, 
ethnic background, verbal and mathematics abilities, and sex of the 
student. The study applied the Hierarchical Linear Model (HLM) to 
analyze data from 4,917 14-year~old students from 233 Australian 
schools; it also incorporated the Rasch analysis model for the 
construction of the science achievement scale. The correlations 
between student characteristics and their science achievement 
indicated that sex, attitude towards science, and ethnicity of the 
student are comparatively weak correlates of science achievement and 
that verbal and quantitative ability and socio-educaLional level had 
much stronger correlations. School effects found to be statistically 
significant in influencing student differences in science achievement 
included the percentage of female teachers in the school, teacher 
decision making, average soc io^educat ional level of students, average 
verbal ability, average quantitative ability, and average ethnicity 
of students. These student and school effects were utilized to 
develop a preliminary integrated model of school effectiveness. No 
statistically significant variations in sex differences in science 
achievement between schools was found. The study concluded that 
results of this preliminary study indicate that further path analysis 
and use of the HLM for all countries in the SISS should provide 
useful information regarding the relationship between student 
achievement, attitudes towards science, and other home and school 
environment characteristics. A list of 29 references is provided, 
(MDH) 
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School Effectiveness and Science Achievement: 
Are there any Sex Differences? 

Deidra J Young and Barry J Fraser 
Science and Mathematics Education Centre 
Curtin University of Technology, Perth. Western Australia 

Introduction 

The purpose of this study was to use secondary analysis of an Australian database known as 
the Second International Science Study to examine the role of student, school and home 
factors in explaining student differences in science achievement, especially sex differences in 
achievement. This study employed methodology which accommodated both the Rasch model 
(science test items were calibrated for item difficulty and student ability) and the multilevel 
nature of the data (students clustered within schools). 

Student characteristics investigated in this study for their influences on science achievement 
included the socio-educational lev^l (home background), attitude towards science, ethnic 
background, verbal and mathematical abilities and sex of the student. School effects were 
also investigated in terms of their ability to explain student differences in science 
achievement. These included aggregated student characteristics, the school type (private 
/public), the sex composition of the school (single-sex/coeducational schools), the local 
population, the ratio of students to teachers, the percentage of female secondary teachers, ttie 
degree of teacher decision making, the involvement of the parent-teacher committee and the 
source of operating costs for the school. The school effects were categorized as contextual or 
process depending on their nature. Contextual school effects reflected the school's physical 
resources, student-body composition and other school environmental factors, whereas the 
process school effects are related to the management and organizational structures of the 
school. In this paper, school effectiveness research is reviewed and then related to an 
integrated model of school effectiveness. Following this, the sample is described and 
methodology summarized. Finally, the relevance and importance of this study are elaborated. 

School Effectiveness Research 

The central proposition of this paper is that school characteristics and the school environment 
can influence student achievement and sex differences in science achievement, and this 
cannct be ignored by researchers in the field of gender research. Previously, there has been 
considerable research emphasis on the ability and family background of the student in 
deteimining academic performance. The Coleman Report (Coleman et al., 1966, p. 296) 
estimated that the percentage of school influence on student achievement was about 10 to 20 
percent of the total variance, yet the methodology used by Coleman did not account for the 
hierarchical nature of students nested in schools. Further, Coleman's study found that the 
American public schools were remarkably homogeneous both in terms of educational 
resources and student outcomes. Unfortunately, this study also demonstrated that large 
differences in student achievement existed between major racial and socioeconomic groups. 
Coleman^s findings were repeated in further large-scale studies (Jencks et al., 1972, 1979; 
Hauser, Sewell & Alwin, 1976), with the resulting school effectiveness research finding that 
(1) school level variables account for relatively small amounts of variance in student 
achievement, (2) physical resources at the school level account for relatively small amounts 
of the variance between schools and (3) student characteristics such as socioeconomic status 
and home background should be used to adjust student achievement in statistical analysis of 
such large-scale studies (Scheerens, 1990). 

While Coleman's research focused on the small effect which school characteristics have on 
student achievement when home background is controlled, more recent school effectiveness 
research has attempted to open the black box of the school environment and examine more 
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closely what happens in schools. The current school effectiveness research genre has shifted 
focus from the black box to opening up within-school characteristics which include school 
organization, culture and educational technology. Case study research has become more 
useful in this respect, as well as large-scale studies such as the International Association for 
the Evaluation of Educational Achievement (TEA) science studies: First International Science 
Study (Comber & Keeves, 1973) and Second International Science Study (Posdethwaite & 
Riley, 1992). This paper describes some analyses of data from the Austrafian database of the 
Second International Science Study (SISS) collected in 1984. In the recent school 
effectiveness research, typified by Mortimore et al. (1988), a combined large-scale study and 
case studies, which found that student progress was enhanced by effective schools 
irrespective of the student's own social class, sex or ethnic group. Moreover, Mortimore 
asserted that: 

Even though overall differences in patterns of pupil attainment are not 
removed in the most effective schools, the performance of all children is 
raised and, as we have demonstrated, disadvantaged children in the most 
effective schools can end up with higher achievements than their advantaged 
peers in the less effective schools. 
(Mortimore, 1988, p. 217) 

Scheerens and Oeemers (1989) have criticised the following five factor model of effective 
schools' characteristics derived from recent research: 

1 . Strong educational leadership, 

2. High expectations of student achievement, 

3. Emphasis on basic skills, 

4. A safe and orderly climate, and 

5. Frequent evaluation of pupils' progress. 

These researchers assert that this model is correlational only, not causal, in that the effects of 
high achievement could lead to a safe and orderly climate and not vice versa. Emphasis on 
basic skills is also a questionable goal because higher-level cognitive learning and affective 
outcomes are equally important. Finally, the independence of these five factors is doubtful; 
frequent evaluation and orderly climate are likely to be a reflection of strong instructional 
leadership (Scheerens & Creemers, 1989, pp. 692-693). 

Scheerens (1990, p. 63) proposed an analytical scheme (Figure 1) to explain indicator 
systems, which incorporates the contextual data (policy, school environment), inputs 
(resources, teacher characteristics), process data (curriculum, school organization and 
climate), ouputs (achievement) and outcomes (employment and income). The use of process 
indicators as an explanation of why some schools do better than others, in terms of student 
achievement, should not be ignored in school effectiveness research contends Scheerens. 

An integrated model of school effectiveness developed by Scheerens (1990, pp. 72-73) was 
used in order to define school- and student-level educational performance indicators. This 
model incorporates an analytic systems model which recognises context, input, process and 
output variables; a multi-level framework discriminating between student and school 
characteristics; perspectives for conceptualising the interrelationships between variables 
defined at different levels (most notably contingency theory and organisational theory that 
incorporates the schools* primary processes); and substantive findings from different types of 
educational effectiveness research (see Figure 2). 
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Context 



e.g. consumer demands 
school environment 
policy measures 
higher administrative level 



Input 



Process 



resources 
teacher- 
qualifications 



cuniculuni 
school organization 
school climate 



Output 



achievenfient/ 

attainment 

measures 



Outcomes 

employment 
earnings 



Figure 1. Context-input-process-output-outcome Model of Schooling 
(Scheerens, 1990, p, 63) 
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COQiCXt 

* achicYCtncnt stiniuJanb from higher administrative levels 

- development of educational consumerism 

- "co-variables" like school size, student-body composition, school category, mtanAuial 



Inputs 

• teacher experience 
- per pupil expenditure 
' parental support 



Process School Le-/el 

- degree of achiev ancnt oriented policy 
' educational leadership 

- consensus, cooperative planning of teachers 

- quality of school curricula in terms of content covered, 
and formal stnicture 

- orderly atmosphcie 



Qassroom Level 

time-on'lask (including homework) 
stmctured learning 
opportunity to I cam 
high expectations of pupils' progress 
degree of cvalualioo and monitoring of pupil's progress 
-reinforcement 



Outputs 

Student Achievement 
adjusted for; 

- previous achicvernent 

- intelligence 

- socioecooomic status (SES) 



Figure 2. Integrated model of school effectiveness. 
(Scheerens, 1990, p. 73) 
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Objectives and Theoretical Framework 

The purpose of this study was to investigate the role of student, school and home background 
factors in explaining differences in science achievement using a databa*>e from the Second 
International Science Study. Additionally, sex differences in science achievement were 
examined with respect to other student and school characteristics. 

Although sex differences in science achievement have been investigated in previous large- 
scale studies (Keeves, 1973; Comber & Keeves, 1973; Kelly, 1978), some of these studies 
assumed that the data had been collected as a simple random sample and therefore ignored 
the stratification of the sample by state/territory and school type (government, Catholic and 
independent), while others neglected the multilevel nature of data from students who are 
clustered within school systems. This study employed methods of analysis which 
accommodated both the complex sample design and the multilevel nature of the data. 

The integrated model of school effectiveness (Figure 2) suggested by Scheerens (1990) 
includes a contextual effect, teacher and parental inputs, school and classroom process 
effects, and student achievement outputs. The student achievement outputs are adjusted for 
previous achievement, intelligence and socioeconomic status. In this study, some of the 
contextual effects examined included the size of the population in tJie school area, the ratio of 
students to teachers and the percentage of female secondary teachers in the school. Schools 
were categorised into three school types: government. Catholic or independent. However, in 
this study, the variable school type was coded in the same way for both Catholic and 
independent schools and differently for government schools. The student body composition 
was characterised by sex composition or single-sex versus coeducational, average socio- 
educatii/jal level, average verbal ability, average quantitative ability, average ethnicity and 
average attitude towards science. Included in the school context was the source of operating 
costs as either private or public. Two process-school level components in this study were the 
parent and teacher committee's involvement in the running of the school and a measure of 
teacher decision making in the school. These variables are defined more fully in Table 1 . 

The output measure of student achievement for this study was a science achievement, a scale 
constructed from 46 science test items. All of the test items were multiple-choice in format 
and covered the three science content areas of biology, chemistry and physics. In addition, 
the items varied in the degree of cognitive ability required (from concrete to analytical). The 
science achievement output of studenis was adjusted for the following student characteristics: 
sex, attitude towards science, ethnicity, verbal ability, quantitative ability and socio- 
educational level. In summary, the integrated model of school effectiveness investigated in 
this study included the following components: 



Type of Indicator Variable Description Variable Name 

Context Size of population of school area areapopn 

S tudent/teacher ratio stutchr 

Percentage of female teachers femtch 

Average socio-educational level avsel 

Average verbal ability averb 

Average quantitative ability avmath 

Average ethnicity minority 

Average attitude towards science avenjsci 

School type schtyp 

Sex composition of the school sexcomp 

S ource of operating costs opcost 

Process Teacher decision making tchdec 

Parent teacher committee involvement ptc 
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Output 



Science achievement 
Sex 

Attitude towards science 
Ethnicity 
Verbal ability 
Quantitative ability 
Socio-educational level 



science 
sex 



enjsci 

etlinty 

tot2v 

toi2q 

sel 



While these school contextual and process effects on student perfoimance in science are 
somewhat limited when compared with tlie Scheerens model (Figure 2), this paper represents 
only a preliminary report Further research into inputs, context, process and ouq)uts using the 
Second International Science Study database will include teacher characteristics and school 
expenditure. 

Description of Sample 

There were 4917 14-year-old students in this sample (2565 girls and 2352 boys) selected 
from 233 schools. The target population of 14-year-old Australian students consisted of 
246 132 students within 2144 schools at the time of this survey (1983/84), The sample design 
used in this study was a stratified two-stage cluster design, with schools selected randomly 
from within each of 24 strata (the eight Australian states and the three school types: 
government. Catholic and independent) and students selected randomly from within each 
school. Table 2A presents a listing of the 24 strata and a description of the schools in each 
strata. The actual sampling statistics are found in Table 2B, with each strata presented by 
targeted population sizes, designed sample sizes and achieved sample sizes. The sampling 
fractions, percentage response rate and stratum weights are also provided here. This complex 
sample design meant that the normal assumptions of simple random sampling could not be 
made validly in order to test statistical sigtiificance. For this reason, a multilevel model was 
developed which accounted for the nested nature of die data. In addition, students in each 
strata were weighted according to the proportion of students whom they were supposed to 
represent. 

Methodology 

The Hierarchical Linear Model 

The use of powerful computers to analyse large databases during the First International 
Mathematics Study focused on the home and school variables influencing student 
achievement, while using the student as the unit of analysis (Husdn, 1967). However, this 
type of analysis did not adequately address die variability of schools contributing to statistical 
tests of significance. Studies in the United States tried to compensate for this problem by 
only looking at tiie school differences; for example, Coleman et al. (1966) used tiie school as 
the unit of analysis. Further, a large database was analysed by Peaker (1967) in England by 
exanunation of between-school means (aggregated student data) and pooling between 
students. However, these studies ignored the differences between-students within- schools 
which may contribute towards explaining the variance. The controversy over the most 
appropriate unit of analysis has continued until the importance of a different approach to 
educational research was fij-st proposed by Cronbach: 

The majority of studies of educational effects - whether classroom experiments, or 
evaluations of programs , or surveys - have collected and analysed data in ways that 
conceal more than they reveaL The established methods have generated false 
conclusions in many studies. (Cronbach, 1976, p. 1) 

Most educational research revolves around students who receive sc. ooling in classrooms 
located within schools, within school districts, within states, etc. The grouping of students, 
classes and schools occurs in a hierarchical order with each group influencing the members of 
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the grou;^ in thought and behaviour. The nature of these hierarcliical structures produces 
multilevel data. Theories about the effects of the multilevel structure of education (the 
different levels of the educational hierarchy) should lead to attempts to specify models v^hich 
involve the analysis of multilevel educational data. Burstein (1980) believes that these 
theories eventually will replace experimental design and analysis with the natural design and 
analyses that evolve from the multilevel structure of data. 

The amount of variation in estimates of variables affecting academic achievement across 
different levels of analysis cannot be ignored by serious educational researchers. Traditional 
linear models on which most researchers rely require the assumption that subjects respond 
independently, yet most subjects are 'nested* within classrooms, schools, districts, states and 
countries so that responses within groups are group dependent To ignore the nested structure 
of this type of data ultimately will give rise to problems of aggregation bias (within-group 
homogeneity) and imprecision (Raudenbush, 1988). 

This Hierarchical Linear Model (HLM or multilevel analysis) provided an integrated strategy 
for handling problems such as aggregation bias in standard error estimates and erroneous 
probability values in hypothesis testing of school effects. For this study, HLM was chosen as 
the model most appropriate to study school and student effects relating to science 
achievement and HLM2 (Bryk, Raudenbush, Seltzer, & Congdon, 1989) was selected as the 
computer package most suited to analyse the large amount of data in SISS. The use of the 
HLM in order to investigate the influence of the organisational structure of the school on 
student performance has been documented by Bryk and Raudenbush (1989, pp. 159-204), Lee 
and Bryk (1989) and Raudenbush and Bryk (1986). The present study sought to examine the 
role of school effects in explaining science achievement and sex differences in science 
achievement. Research on school effects is described as a set of data analysed at the 
individual student level, with the assumption that classrooms and schools affect students 
equally. However, when the effects vary among individuals and their contexts, this type of 
statistical analysis can be misleading (Bryk & Raudenbush, 1987), Ordinary least squares 
analysis provides infoimation about the total variance, but does not break this total variance 
into the between- and within-classroom effects. This study endeavoured to explain variations 
in student outcomes by first decomposing observed relationships into between- and within - 
school components. 

The analysis of gender differences in science achievement and attitudes between different 
school types must take into account variations from school to school unless inferences are to 
remain doubtful. Raudenbush and Bryk (1986) point out the fallacies of research findings 
which ignore the potential effects of the school or classroom as sociological units, citing 
many research studies with doubtful inferences. These researchers introduced the concept of 
the hierarchical linear model (HLM), which accommodated both school and student level 
differences. Their reanalysis of data from a random sample of United States high schools 
illustrates technical and conceptual advances facilitated by HLM and showed that the 
relationship between socioeconomic status and mathematics achievement varied substantially 
across US high schools and that much of this variation was attributable to school type (public 
versus Catholic). Distinguishing between micro parameter variance (such as school or 
classroom) and the sampling variance was possible with HLM, which makes it possible to 
partition the socio-educational level effect into within- and between-group components which 
yielded an estimate of the school type effect substantially different from earlier estimates. 
Similarly, Lee's (1986) reanalysis of data from High School and Beyond study (Coleman et 
al., 1966; Haertel et al., 1987) revealed that differences between public and Catholic schools 
were attributable to the curriculum and the discipline policies of the schools. 

Rasch Analysis 

In addition to using multilevel analytic techniques, this study also incorporated the Rasch 
analysis model for die construction of the science achievement scale (Rasch, 1960). Rasch's 
models for measurement are based upon Thurstone's models (1925, 1926, 1927, 1959) which 
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were independent of the data. The Rasch model, was fiirther developed into a function of the 
difference between parameters p^for person ability and d( for item difficulty (p^,- d(): 

Prlpositive response} = ^ Equation 1 

For each student, 46 science achievement items were selected and a Rasch calibration 
produced an ability estimate for all 46 items (based upon equation 1). The 46 science test 
items consisted of 20 common core test items answered by all students and 26 rotated test 
items responded to by different subgroups of students. The purpose of the Rasch analysis 
was to make the science achievement scores independent of the sample and the item 
difficulty; some of the rotated test items may have been more difficult than others. The 
science test items also represented biology (20 items), chemistry (12 items) and physics (14 
items) content areas. The calibration was made using the Rasch model, which is independent 
of student ability and item difficulty, and TITAN (Adams & Toon, 1991), a Rasch analysis 
computer software package for calibrating items according to equation L 

In Table 3, the frequency of correct scores, item infit meansquares and t statistics are 
provided. In particular, large infit meansquares (greater than 1.00) are indicative of a 
misfitting item. This table shows that there were no items which misfit i.e. there were no 
items much greater than 1.00. Further statistical results are provided in Tables 4 and 5. 
Table 4 provides a summary of item estimates and case estimates. Note that the reliability of 
the case estimate was 0.82. The graph of item estimates on all students is presented in Table 
5 depicting the spread of items and students. This graph reveals that there is probably a lack 
of more difficult items (towards the upper part of tiie graph) in this set of science test items. 

Results and Discussion 



The correlations between student characteristics and theu* science achievement is described in 
Table 6 and reveals that sex, attitude towards science and ethnicity of the student are 
comparatively weak correlates of science achievement, altiiough statistically significant. The 
other three student level variables, verbal and quantitative ability and socio-educational level, 
had a much stronger relationship which was also significant. When the variance in student 
science achievement was decomposed into school (between-school) and student (within- 
school) components using the hierarchical linear model (HLM), 15.2 percent of die total 
variance was found to be related to between-school differences, while 84.8 percent was 
related to witiiin-school differences (Table 7). The total between-schoois variance of 0.179 
and within-schools variance of 0.998 resulted in an intra-class correlation of 0.152. 



The partitioning of variance in science achievement among students into the within- and 
between-school components was achieved using the HLM2 computer package (Bryk, 
Raudenbush, Seltzer & Congdon, 1989). A random average science achievement estimate 
was specified for tlie within-school model: 

Scienceij = Pqj + Rij 

where / = 1, . . students in school;,; = 1, . . / schools, Sc/^wce/y represents science 

achievement of student / in school ;, j3(;)y represents the mean science achievement for school ; 
and Rij represents random error of student / in school ;. 

An unconditional between-school model was also specified: 



Poj = l^j + Uj 

where Pqj represents the mean science achievement for school ;, /ly is equal to j3Qj and Uij 
represents random error of school ;. 
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The hierarchical linear model was used to include student-level variables to the above model 
and exaniine the reduction in unexplained variance. The addition of sex to the model reduced 
the within-school variance by 1.7 percent, while the addition of all of the student 
characteristics described previously reduced this variance by 43.4 percent (a further 41.7 
percent). While sex, ethnic and social differences were notable in their effects on student 
science achievement, adjustments for these effects were made and then school effects added 
to the hierarchical linear model 



Scienceij = Pqj + fijjSexij + fizj^njmij + ^sfthntyij + p4jTot2yij + ^sfotlqij + ^^jSelij + R^j 

where Science represents student science achievement, Sex represents sex of the student, 
Enjsci represents the student's attitude towards the science, Ethnty represents the student's 
ethnic background, Totlv represents the verbal ability of the student and Totq represents the 
quantitative ability of the student and Sel represents socio-educational level of the student. 
The beta coefficients are described as follows: 

^Qj = Mean science achievement for students in school j. 
Pjj = The degree to which sex differences in science achievement related to their 
achievement. 

= The degree to which attitude towards science differences among students in 

school j is related to their science achievement. 
= The degree to which ethnic differences among students in school j is related to 

their science achievement. 
P4j = The degree to which differences in verbal abilities among students in school j is 

related to their science achievement. 
/Jjy = The degree to which differences in quantitative abilities among students in school 

j is related to their science achievement. 
/J(jy = The degree to which socio-educational differences among students in school j is 

related to their science achievement. 

The between-schools variance was reduced markedly by 76.1 percent when school effects 
were added to this model. In Table 8, school effects were found to be statistically significant 
in influencing student differences in science achievement including percentage of female 
teachers in the school (negative small effect), teacher decision making (negative small effect), 
average socio-educational level of students (large positive effect), average verbal ability 
(large positive effect), average quantitative ability (very large positive effect) and average 
ethnicity of students (small positive effect). In contrast, there was evidence that some of the 
school effects investigated which were not useful in explaining student differences in science 
achievement, such as the size of the population in the area of the school, existence of parent 
teacher committees, operating costs, average student attitude towards science, school type and 
sex composition of the school. Student differences in science achievement were explained by 
the following school effects model and found to fit well, that is, these school level variables 
were associated with greatly reduced school level unexplained variance and were statistically 
significant in accounting for student differences in science achievement. 

^Oj = 700 + YioFemichj + Y2oTchdecj + Y^oAvselj + y4oAverbj + y^QAvrmthj 
+ Y60^inontyj + Uqj 

Plj = 770+ i^Jj 
P2j = 720 + ^2j 
P3j = rjO + ^3j 
P4j = Y40 + 

P6j = 760 + 
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These student and school effects were statistically significant in explaining student 
differences in science achievement and a model of school effectiveness incorporating these 
effects is provided in Figxrre 3. It should be emphasised that this study is preliminary and 
further effects are being studied for incorporation into the model. Of particular interest are 
the teacher characteristics and other process school effects. 

The relationship between these effects and sex differences was examined by freeing up the 
variable sex (.allowing the student variable sex to vary between schools). However, there 
were no statistically significant variations in sex differences in science achievement between- 
schools. Sex differences were found to contribute relatively little towards explaining 
differences in science achievement, with more significant effects related to verbal and 
quantitative abilities and social class. Moreover, student characteristics other than sex 
explained 41.7 percent of tlie student differences, while school level differences accounted for 
15.2 percent. The higher male science achievement (/ = -11. 02, p - 0.000) was significant, 
but offset by otlier effects to a greater extent. 



Context 

- peiccnUgc of female teadim 

- average jludcnl chaTiicfyistics 

average sociocducalional level 
average vtirb A ability 
Average quantitative ability 
average ethnicity 



Process School Level 

- degree of teacher decision making involvement 




Outputs 

Student Achievement 
adjusted foe: 
■ sex 

- attitude towards sciaice 

- ethnicity 

- verbal ability 

- qua^idtativc ability 

- sociocducationa] level 













Figure 3. Preliminary Integrated Model of School Effectiveness 



Importance of this Study 

This analysis of the Australian student database, the Second International Science Study 
(SISS), established the relative importance of school-level and student-level variables in 
explaining student differences in scie^^^e achievement. A combination of Rasch analysis and 
the hierarchical linear model (HLN ' was used to demonstrate the relationship between 
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contextual and process school factors and student achievement and how they are inextricably 
woven into the student's own characteristics, ^hile some school effects were more useful 
than others in explaitiing student differences in science achievement, the significance of the 
aggregated student level variables cannot be ignored. While most of iiie unexplained variance 
was at the student-level, student variables reduced this by half. Stuaent characteristics found 
to be most significant were sex of the student, attitude towards science, ethnicity, verbal 
ability, quantitative ability and socio-educational level. Similarly, the smaller portion of 
school-level unexplained variance was significantly reduced by the school-level variables. 
School-level variables found to be most significant were percentage of female teachers (-ve), 
teacher decision maJdng (-ve), average ability (-fve), average attitude (-fve), average ethnicity 
(+ve) and average socio-educational level (+ve). The peer effect appeared to be greatest for 
average student ability. In summary, while specific school and teacher characteristics were 
notably influential, they appeared relatively weak in comparison to the student's own personal 
characteristics and the characteristics of the student's own peers. Additionally, ability 
measures consistently showed high correlations with science achievement, as did the socio- 
educational level of the student 

Although the present study is basically a preliminaiy one, further path analysis and use of the 
hierarchical linear model for all countries participating in tlie SISS should provide useful 
information regarding the relationship between student achievement, attitudes towards 
science (as attitude is used as an outcome variable) and other home and school environment 
characteristics. 
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Table 1. Description of Student and School Level Variables 



Student Level Variables 



Science Science achievement score calibrated for student ability and item difficulty 

using the Rasch model 
- 46 multiple-choice items 



Sex 
Enjsci 

Ethnty 



Sex of student 

Attitude towards science scale 
- 16 items 

Ethnicity composite scale 
- home language 



- student's country of birth 

- father's country of birth 

- motiier's country of birth 



1 = Male 

2 = Female 
1 = agree 

0 = disagree or not sure 

1 = English 

0 = not English 

1 = Australia, UK, Ireland, USA, 

Canada, New Zealand 
0 = Europe, Asia, South America, 
Central America, Africa, 
Oceania 



Tot2v Verbal ability 

Tot2q Quantitative ability 

Sel Socio-educational level composite scale 

- father's education & occupation 

- motiier's education & occupation 

- number of books in the home 



School Level Variables 



Schtyp 

Sexcomp 

Areapopn 
Stutchr 



School type 
- government or Catiiolic/independent 

Sex composition of the school 



1 = Private a, C) 

0 = Public (G) 

1 = Single-sex 

0 = Coeducational 



Approximate population of the area where your school is located 
Ratio of number of students to number of teachers in school 



Femtch Percentage of female secondary teachers in school 

Tchdec Teachers responsible for decision 1 = Teachers responsible 

making in the school composite scale 0 = Teachers not responsible 

- range of science subjects 

- content of science courses 

- choosing textbooks for science students 

- selectmg science equipment 

- selecting students for entrance to the school 

- determining size of tuition fees 

- deciding on major expenditure (e.g. new buildings) 

- determining conditions of employment for teachers 

- selectmg teachers for the school 

- selectmg the school principal 

- making rules and regulations for the students 

1 r- 
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?tc 



Opcost 

Avsel 

Averb 

Avmath 

Miiiority 

Avenjsci 



Involvement in parent teacher committee (ptc) activities 

- school has one or more committees on which 1 = ptc involved 
both parents and teachers are formally represented 0 = ptc not involved 

- social work in the community 

- social and cultural activities 

- activities for raising money for the school 

- discussions about particular curriculum and instmctional matters 

- dfxussions about general policies for school organization or curriculum 

- organization of parent education activities 

- selection of teaching staff 

- decisions about school expenditure 

- organization of science-related activities 



Source of operating costs for the school 



1 = govemment (public) 

2 = partially government 

3 = private 



Average socio-educational level of students attending the school 

- aggregated variable from smdent Sel 

Average verbal ability of students attending the school 

- aggregated variable from smdent Tot2v 

Average quantitative ability of students attending the school 

- aggregated variable from student Tot2q 

Average ethnic composition of students attending the school 

- aggregated variable from smdent Ethnty 

Average attitude towards science of students attending the school 

- aggregated variable from student Enjsci 
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Table 2. Sample Size and Composition. 
Table 2. A. Australian Stratification: Population 2 stratum definitions (14-year-old students) 



Stratum (h) Description 



14 Australian Capital Territory (ACT) - Government Schools 

1 5 Australian Capital Tenitory (ACT) - Catholic Schools 

16 Australian Capital Territory (ACT) - Independent Schools 

24 New South Wales (NSW) - Government Schools 

25 New South Wales (NSW) - Catholic Schools 

26 New South Wales (^^SW ) - Independent Schools 

34 Victoiia (Vic) - Government Schools 

35 Victoria (Vic) - Catholic Schools 

36 Victoria (Vic) - Independent Schools 

44 Queensland (Qld) • Government Schools 

45 Queensland (Qld) - Catholic Schools 

46 Queensland (Qld) - Independent Schools 

54 Southj^ijufigalia (S A) - Government Schools 

55 SoutiiAultralia (SA) - Catholic Schools 

56 South Australia (S A) - Independent Schools 

64 Western Australia (WA) - Govemxnent Schools 

65 Western Australia (WA) - Catholic Schools 

66 Western Australia (WA) - Independent Schools 

74 Tasmania (Tas) - Government Schools 

75 Tasmania (Tas) - Catholic Schools 

76 Tasmania (Tas) - Independent Schools 

84 Northern Territory (NT) - Government Schools 

85 Northern Territory (NT) - Catholic Schools 

86 Northern Territory (NT) - Independent Schools 



Table 2.B. Sampling Statistics Summary 



Target Population Designed Sample Achievp i Sample Response Stratum | 

Stratum Schools Students Schools Students School* Students Schools Weight ' 

h A{h) N(h) a(h) n(h) a'(h) n'(h) f(h) % v/(h) 



14 


16 


2609 


10 


240 


6 


127 


0.049 


60 


0.4104 


15 


6 


889 


3 


72 


2 


46 


0.052 


67 


0,3861 


16 


3 


245 


2 


48 


2 


39 


0.159 


100 


0.1255 


24 


421 


65453 


33 


792 


28 


568 


0.009 


85 


2.3020 


25 


158 


14505 


6 


144 


3 


65 


0.004 


50 


4.4580 


26 


55 


3607 


8 


192 


7 


155 


0.043 


88 


0.4649 


34 


410 


49544 


28 


672 


23 


475 


0.010 


82 


2.0837 


35 


131 


11694 


8 


192 


6 


130 


0.011 


75 


1.7970 


36 


80 


5886 


7 


168 


7 


164 


0.028 


100 


0.7170 


44 


198 


28998 


30 


720 


28 


660 


0.019 


93 


1.0345 


45 


86 


6724 


7 


168 


7 


157 


0.023 


100 


0.8656 


46 


32 


2737 


6 


144 


6 


137 


0.050 


100 


0.3991 


54 


143 


19439 


34 


816 


30 


605 


0.031 


88 


0.6419 


55 


77 


2239 


6 


144 


3 


69 


0.031 


50 


0.6482 


56 


18 


1330 


4 


96 


4 


as 


0.066 


100 


0.3019 


64 


129 


17308 


32 


768 


28 


606 


0.035 


88 


0,5706 


65 


46 


3069 


6 


144 


5 


115 


0.037 


83 


0.6331 


66 


30 


1466 


4 


96 


3 


70 


0.048 


75 


0.4184 


74 


70 


6725 


22 


528 


17 


368 


0.064 


77 


0.3108 


75 


11 


860 


4 


96 


4 


86 


0.101 


100 


0.1974 


76 


11 


446 


3 


72 


3 


55 


0.124 


100 


0.1616 


84 


10 


1132 


10 


240 


9 


IBS 


0.166 


90 


0.1203 


85 


2 


212 


2 


48 


1 


22 


0.104 


50 


0.1925 


86 


1 


26 


1 


24 


1 


22 


0.846 


100 


0.0236 


Total 


2144 


246132 


276 


6624 


233 


4917 


0.020 


84 





n'«4917 
N - 246132 
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Table 3. Construction of Science Acliievement Estimate using the 
Rasch Model and the Computer Software Package: TITAN* 



Item Estimates (Thresholds) In Input Order all on Science (N = 4917 L = 46) 



Item Name 


Score 


Maximum 


Threshold 


Infit 


Outfit 


Infit 


Outfit 






Score 


1 


Meansq 


Meansq 


t 


t 


P2m8 


3356 


4801 


-.26 


1.02 


1.02 


1.3 


.7 


P2m9 


3550 


4878 


-.44 


1.00 


1.00 


.3 


A A 

0.0 


P2ml0 


2970 


4779 


.14 


LOO 


1.00 


.1 


1 

.1 


P2mll 


1024 


4773 


2.30 


1.09 


1,25 


4.0 


6.0 


P2ml2 


1736 


4790 


143 


1,02 


1.06 


1.6 


2.7 


P2ml4 


3307 


4886 


-.15 


1.00 


1.02 


,3 


.7 


P2ml5 


4089 


4886 


-1.19 


,92 


,87 


-3.2 


-2.8 


P2ml6 


3545 


4858 


-,45 


,93 


.93 


-4.1 


-2.1 


P2ml8 


2247 


4848 


.93 


,96 


.96 


-3,5 


-2.2 


P2ml9 


3874 


4854 


-,89 


.95 


.91 


-2.4 


-2.1 


P2m20 


1693 


4367 


1,30 


.96 


.96 


-3.0 


-1.6 


P2m21 


4252 


4908 


-1.46 


.92 


.92 


-2.8 


-1.4 


P2m22 


3943 


4901 


-.94 


1.04 


1.14 


2.0 


3.1 


P2m24 


1452 


4739 


1,74 ^ 


.99 


1.08 


-.6 


O A 

2.9 


P2m25 


1839 


4675 


1.29 


1.05 


1.11 


3.2 


C A 

5.0 


P2m26 


3683 


4897 


-,58 


1.06 


1.19 


3.3 


5.1 


P2m27 


3768 


4899 


-.69 


,97 


.92 


-1,6 


-i.i 


P2m28 


4247 


4901 


-1,46 


1.02 


1.08 


.6 


1 A 

1.4 


P2m29 


2208 


4663 


.87 


31 


.97 


-2,2 


1 c 


P2m30 


2672 


4816 


.48 


1.04 


1.05 


O A 

3.0 


2.3 


P2al 


1875 


2459 


-.69 


.93 


.88 


o o 
-2.5 


-2.4 


P2a2 


1442 


2443 


.26 


1.00 


1.01 


,1 


.3 


P2a3 


1313 


2459 


.55 


1.02 


1 AC 

1.05 


1 o 

1.2 


1 n 
1.9 


P2a4 


2285 


2470 


-2.24 


.93 


T7 
.11 


-1,1 


-Z.U 


P2a7 


1893 


2465 


-.72 


1,09 


1.25 


3.4 


A A 

4.4 


P2a9 


1610 


2453 


-.07 


1.04 


1.03 


1 A 

1.9 


A 

.9 


P2al0 


1240 


2382 


.63 


1.01 


1,03 


.6 


1 A 

1.0 


P2bl 


1674 


2391 


-.25 


1.08 


1.12 


3.6 


2.6 


P2b2 


1799 


2420 


-,50 


1.00 


1.07 


A A 

0.0 


1.5 


P2b3 


803 


2367 


1,57 


,94 


1.00 


-2.7 


.1 


P2M 


2131 


2448 


-1.46 


.99 


1,29 


-.2 


3.2 


P2b5 


1998 


2451 


-,99 


1.00 


1,05 


A A 

0.0 


o 
.0 


P2b7 


2152 


2454 


-1.53 


1.03 


1 OA 

1,20 


.6 


0 0 
Z.Z 


P2bl0 


1798 


2447 


-.46 


.97 


.96 


-1.1 


o 


P2c3 






Jo 


.VO 




-Z.J 


-1 4 
-1.*+ 


JrZCo 




94'^4. 


79 




1 05 


1.6 


1.9 


P2c9 


1722 


2431 


-.33 


.94 


.89 


-2,8 


-2.5 


P2clO 


1757 


2437 


-.40 


.94 


.88 


-2,5 


-2.6 


P2dl 


1531 


2432 


.12 


1,01 


1.01 


.4 


.2 


P2d2 


1455 


2434 


.29 


.96 


.92 


-2.1 


-2.7 


P2d3 


1386 


2410 


.41 


1.02 


1.06 


.9 


1.9 


P2d4 


1576 


2435 


.03 


.96 


.93 


-1.9 


-2.1 


P2d7 


1028 


2425 


1.14 


1.07 


l.ll 


3.6 


3.8 


P2d8 


931 


2433 


1.36 


1,00 


1.00 


-.2 


0.0 


P2d9 


1822 


2429 


-.56 


.95 


.90 


-1.8 


-1.9 


P2dl0 


1418 


2428 


.35 


.95 


.96 


-2.9 


-1.5 


Mean 






0.00 


.99 


1.02 


-.2 


.5 


SD 






1.00 


.05 


.11 


2.2 


2.4 



* Titan is a Rasch item analysis computer software program developed by Dr Raymond Adams and Khoo Siek Toon at 
the Australian Council for Educational Research and generously supplied to the authors for this study. 
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Table 4. Summary of Rasch Analysis of Science Test Items 



Total Test Items 



Summary of Item Estimates 

Mean 0.00 
SD 1.00 
SD (adjusted) LOO 
Reliability of estimate 1 .00 

Separation Index 2L86 



Fit Statistics 



Infit Mean Square Outfit Mean Square 

Mean ,99 Mean 1.02 

SD ,05 SD .11 

Infit t Outfit t 

Mean -.22 Mean .45 

SD 2.24 SD 2.41 



Total Students 



Summary of Case Estimates 

Mean 0.78 
SD 1.09 
SD (adjusted) 0,99 
Reliability of estimate 0.82 
Separation Index 2.15 



Fit Statistics 



Infit Mean Square Outfit Mean Square 

Mean 1.00 Mean 1.02 

SD ,17 SD ,39 

Infit t Outfit t 

Mean .01 Mean .05 

SD 0.88 SD .94 



Young and Fraser 1992 



Table 5. Graph of Student Ability Estimates of Science Test Items 

Item Estimates (Thresholds) on Science (N = 4917 students L = 46 items) 

Ability Number of Items 
Estimate Students 

4,0 X 

XX 



3,0 



-1,0 



-2.0 



-3.0 



X 
XXX 

XXXX 
XXXX 
XXXX 

XXXX 11 



XXXXXXXX 

2,0 XXXXXXX 

XXXXXXX 



1.0 



XXXXXXXXXXXXXXX 


24 






XXXXXXXXXXXXX 


43 






XXXXXXXXXX 


12 


68 




XXXXXXXXXXXXX 


20 


25 




XXXXXXXXXXXXXXXX 


67 






XXXXXXXXXXXXXXXXXXXX 


18 


29 




XXXXXXXXXXXXXXXXX 


53 


58 




XXXXXXXXXXXXXXXXXX 


33 


40 




XXXXXXXXXXXXXXXX 


30 


63 


70 


XXXXXXXXXXXXXXXXXX 


32 


62 




XXXXXXXXXXXXXXXX 


10 


61 


64 


XXXXXXXXXXXXXX 


14 


39 




XXXXXXXXXXXX 


8 


41 




xxxxxxxxxxx 


9 


16 


50 


XXXXXXX 


26 


42 


69 


XXXXXXX 


27 


31 


37 


XXXXX 


' 19 


22 




XXX 


45 






XX 


15 






XX 


21 


28 


44 


X 


47 







Each X represents 1 7 students 



34 



59 60 



2u 
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Table 7. HLM Analysis of Variance in Science Achievement 



Science Achievement 



Model 


Between-Schools 
Variance 


Within- Schools 
Variance 


Science Achievement 


0.179* (15.2%) 


0.998 (84.8%) 


Science Achievement + Sex 


0.180* 


0,981 






(1.7% reduction) 


Science Achievement 


0.045* 


0.565 


+ student levelt + school level* variables 


(76,1% reduction) 


(43.4% reduction) 



* Statistically significant variables at p = 0.001 level 

t Student level variables = sex, enjsci, ethnty, tot2v, tot2q, sel 

* School level variables = arcapopn, stutchr, femtch, tchdec, ptc, opcost, avsel, averb, 

avmath, minority, avenjsci, schtyp, sexcomp 



2o 
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Table 8. HLM Analysis of School Effects on Science Achievement 



The Ganuna(*)-Standard En-or-T Statistic Table 





VJdJTUTlai^ ) 


OlallUalLl UliiJi 


1 olaUdUC 


X^*" V OiUC 


For Base Coef. 










Base 


-2.38 


.35 


-6.80 


.000 


Areapopn 


-.02 


.02 


-K24 


.216 


Stutchr 


.00 


.00 


.04 


.969 


Femtch 


.00 


.00 


-2.27 


.023 


Tchdec 


-.28 


.16 


-1.73 


.083 


Ptc 


.04 


.07 


.55 


.580 


Opcost 


.16 


.13 


1.30 


.195 


Avsel 


.27 


.10 


2.73 


.007 


Averb 


.04 


.01 


3,50 


.001 


Avmath 


.14 


.02 


9.09 


.000 


Minority 


.33 


.19 


1.76 


079 


Avenjsci 


.29 


.30 


.98 


.yii 


Schtyp 


-.12 


.13 


-.92 


.361 


Sexcomp 


.04 


.06 


.64 


.519 


For Sex Slope* 










Base 


-.29 


.03 


■11.02 


.000 


For Enjsci Slope* 










Base 


.54 


.05 


9.91 


.000 


For Ethnty Slope* 










Base 


.16 


.06 


2.57 


.010 


For Tot2v Slope* 










Base 


.05 


.00 


18.41 


.000 


For Tot2q Slope* 










Base 


.12 


.00 


34.84 


.000 


For Sel Slope* 










Base 


.29 


.03 


9.46 


.000 


* - The Residual Variance For This Paiameler Has Been Set To Zero. 








Chi Square Table 






Estimated 


Parameter 


Degrees 






Parameter 


Variance 


Of Freedom 


Chi Square 


P-Value 


Base Coef 


.04497 


219 


559.77 


.000 



Sigma squared = 0.565 
Parameter Reliability Estimates: 
Base Tau(l,l) / D-bar(l,l) = 0.597 
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